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Comment  on  Neider:  The  Issue  of  Interpretation  of  Experiments 


ABSTRACT 

It  is  argued  that  Neider' s  study  claiming  to  show  the  benefi¬ 
cial  effects  of  participation  on  store  sales  is  open  to  other  inter¬ 
pretations,  as  are  many  studies  of  participative  decision  making. 
Neider 's  study  actually  had  four  independent  variables:  participa¬ 
tion;  a  sales  plan;  goal  setting;  and  incentives.  The  first  two 
constituted  one  condition;  the  second  two  another;  and  all  four 
another.  Thus  no  difinitive  interpretation  of  the  results  is 
possible.  However,  an  interpretation  that  is  equally,  if  not  more, 
plausible  than  Neider' s  is  that  goal  setting  and  incentives  do  not 
improve  performance  unless  there  is  an  adequate  action  plan. 
Participation  may  be  one  method  of  developing  and/or  gaining  commit¬ 
ment  to  such  plans.  The  general  issue  of  interpretation  of 
experiments  is  discussed. 
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Comment  on  Neider:  The  Issue  of  Interpretation  of  Experiments 

Neider's  (1980)  recent  article  is  symptomatic  of  a  widespread 
problem  in  the  social  sciences:  that  of  non-objective  interpre¬ 
tations  of  research  studies.  This  seems  to  occur  most  commonly 
when  a  pre-existing  ideology,  or  philosophy  of  science,  or  set  of 
assumptions  clashes  with  the  actual  procedures  used  or  results  ob¬ 
tained  by  one  or  more  studies.  For  example,  concurrent  correla¬ 
tions  obtained  between  supervisory  style  and  worker  attitudes  or 
performance  were  traditionally  interpreted  as  indicating  that 
the  first  caused  the  second,  because  a  one-way  causal  model  was 
assumed  in  which  supervisors  controlled  their  subordinates.  Later 
experiments  demonstrated  that  subordinate  actions  could  affect 
supervisory  style  (e.g.,  see  Hollander,  1978).  Now  it  is  recognized 
that  supervisor- subordinate  influence  is  reciprocal.  Theorizing  on 
parent-child  influence  has  gone  through  a  similar  evolution. 

Studies  reported  as  demonstrations  of  organizational  behavior 
modification  have  been  shown  to  have  almost  nothing  to  do  with 
behavior  modification  principles  (Locke,  1977,  1980) .  The  original 
interpretations  simply  reflected  the  authors'  predilection  for  the 
philosophy  of  behaviorism. 

Nowhere  is  the  existence  of  bias  more  evident  than  in  the 
studies  of  participation  in  decision-making.  This  was  documented  in 
Locke  and  Schweiger's  (1979)  major  review  article  (which  was  not 
cited  by  Neider,  1980) .  They  found  that  it  was  common  in  experiments 
on  participation  to  manipulate  anywhere  from  two  to  as  many  as  nine 
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independent  variables  and  yet  to  attribute  the  positive  results  obtained 
mainly  or  solely  to  the  effects  of  participation.  More  recently, 

Bartlem  and  Locke  (in  press)  have  shown  that  even  the  classic  Coch 
and  French  (1948)  article  was  severely  confounded  and  provides 
highly  equivocal  evidence  for  the  benefits  of  participation. 

Similarly,  I  believe  that  Neider's  (1980)  experiment  is  open  to 
interpretations  other  than  the  one  she  made.  To  demonstrate  this, 
let  me  first  summarize  the  studv.  its  purpose  was  to  increase  sales 
in  retail  stores.  There  were  four  experimental  conditions,  each 
confined  to  a  different  store. 

Participation  only  (Store  C) :  employees,  after  discussions  among 
themselves,  developed  an  alternative  planfor  dealing  with  customers 
as  they  entered  the  store.  Instead  of  sticking  with  a  customer 
the  whole  time,  they  suggested  greeting  the  customer,  offering  their 
help  if  needed,  and  then  allowing  the  customer  to  browse  freely. 
Incentive  only  (Store  B) :  This  condition  actually  involved  both 
setting  (minimum  and  midrange)  goals  or  sales  targets  for  each  em¬ 
ployee  and  rewarding  each  based  on  the  degree  of  goal  achievement. 

The  incentive  plan  was  designed  with  employee  input  and  was 
arranged  in  a  three-tiered  hierarchy,  although  we  are  not  told  just 
what  was  offered  at  each  level. 

Combined  condition  (Store  D) :  This  group  was  labeled  "the"  experi¬ 
mental  condition  in  the  abstract  and  on  p.  432,  although  all  the 
conditions  were  called  "experimental  conditions"  on  p.  430.  This 
condition  was  a  combination  of  the  previous  two  conditions. 

Apparently  the  sales  approach  decided  upon  was  the  same  as  in  the 
Participation  only  condition.  While  the  actual  sales  targets  were 


apparently  different  from  those  of  the  incentive  only  groups, 
the  two  sets  of  goals  were  equated  for  subjective  difficulty.  We  are 
not  told  whether  the  content  of  the  incentive  plans  in  Store  B  and 
D  were  the  same  or  different. 

Control  Condition  (Store  A) :  This  store  had  no  treatment. 

The  results  showed  that  the  combined  condition  led  to 
consistently  higher  sales  and  higher  effort  than  the  other 
conditions  over  a  4  week  period.  This  basic  result  was  replicated 
in  a  second  study  in  which  Store  A  was  given  the  same  treatment 
as  Store  D,  while  the  original  treatments  in  Stores  B,  C  and  D 
were  terminated. 


Neider* s  Interpretation 

Neider  interpreted  the  results  in  terms  of  a  contingency  model 
of  participation;  she  argued  that  participation  improves  effort 
and  performance  only  when;  (a)  it  clarifies  the  effort-reward 
relationship;  and  (b)  high  performance  is  rewarded. 

Neider  (1980)  found  that  participation  in  Stores  C  and  D 
increased  the  employees ' understanding  of  how  to  perform  well, but  she 
presented  no  evidence  regarding  perceptions  of  the  effort-reward 
relationship  nor  regarding  the  effect  of  the  incentives  on  the  per¬ 
ceived  valence  of  high  performance. 

Alternative  Interpretation 

Observe  first  that  there  are  actually  four  rather  than  two 
independent  variables  in  this  study  (not  including  combinations) . 
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The  Participation  Condition  included:  (1)  participating  in  the 
development  of  an  action  plan  for  approaching  customers;  and  (2)  the 
plan  itself  (e.g.,  its  quality).  The  Incentive  Condition  included 
two  additional  variables:  (3)  setting  minimum  and  mid-range  goals 
or  sales  targets;  and  (4)  the  offering  of  incentives  for  attaining 
those  goals.  The  combined  condition  included  all  four  variables. 

This  makes  it  impossible,  of  course,  to  definitively  identify  the 
key  variable  or  combination  of  variables  responsible  for  the 
results.  If  there  were  2  levels  of  each  variable,  16  conditions 
would  be  required  to  test  all  possible  combinations--an  impossible 
requirement  for  a  controlled  field  study.  Given  the  results  ob¬ 
tained,  however,  an  alternative  interpretation  is  that:  goals  plus 
incentives  for  goal  attainment  only  lead  to  improved  performance 
when  a  clear  and  effective  action  plan  is  developed  for  reaching 
those  goals.  This  interpretation  is  consistent  with  findings  in 
the  goal  setting  literature  (Locke,  Shaw,  Saari  and  Latham,  in  press) . 
While  not  specifying  the  role  of  participation,  this 
interpretation  seems  at  least  as  plausible  as  the  one  offered  by 
Neider  (1980)  which  ignored  the  possible  role  played  by  goals. 

In  certain  respects  the  present  interpretation  may  be  more 
plausible  than  Neider's  (1980).  For  example,  Locke,  Feren, 

McCaleb,  Shaw  and  Denny  (1980  )  found  that  field  studies  of 

money  incentives  and  goal  setting  have  led  to  median  performance 
increases  of  30%  and  16%  respectively,  while  field  studies  of 
participation  have  yielded  a  median  increase  of  only  0.5%.  This 
suggests  that  incentives  and  goals  may  be  more  potent  variables 
than  participation  with  respect  to  motivation. 
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There  are  also  logical  grounds  for  treating  money  and  goals 
as  more  fundamental  than  participation.  Without  pay,  workers 
would  not  show  up  for  work  at  all  (although  they  will  certainly 
do  some  work  on  non-incentive  pay) .  But  without  at  least  im¬ 
plicit  goals  (e.g.,  job  descriptions  or  assignments)  effort  would 
not  be  directed  toward  anything.  In  contrast,  many  situations 
exist  where  work  gets  done  without  subordinate  participation. 

Locke  and  Schwieger  (1979) ,  however,  have  argued  that 
participation  may  be  quite  useful  from  a  cognitive  viewpoint,  i.e., 
as  a  means  of  obtaining  ideas  which  will  lead  to  better  quality 
decisions.  In  the  present  study  the  probable  benefit  of  participation 
was  that  it  led  to  the  development  of  a  more  effective  plan  for 
dealing  with  customers.  (Neider  offered  no  reasons  as  to  why  the 
employees'  plan  was  superior  to  the  store's  own  policy,  but  one  can 
surmise  that  customers  who  are  followed  around  until  they  buy  some¬ 
thing  feel  harassed  and  probably  spend  less  time  looking  for 
merchandise  than  those  who  are  allowed  to  browse  at  will) .  Thus  a 
second  part  to  the  above  interpretation  of  this  study  could  be  that: 
better  quality  action  plans  will  be  developed  by  employees  working 
participatively  than  by  management  working  unilaterally.  This 
statement  obviously  requires  qualification;  its  validity  will 
depend  on  the  relative  knowledge  and  expertise  of  employees  and  manage¬ 
ment  regarding  the  issue  at  stake  (Locke  and  Schwieger,  1979)  . 

An  unanswered  question  from  Neider ' s  study  is:  What  would  have 
happened  if  the  employee's  plan  had  been  imposed  on  clerks  in 
another  store  (e.g., 


as  a  new  store  policy)?  If  the  plan  had  worked 
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equally  well  (assuming  goals  and  incentives  were  also  present) ,  this 
would  indicate  that  the  benefits  of  participation  were  mainly 
cognitive  (i.e.,  the  content  of  the  ideas)  rather  than  motivational 
(i.e.,  commitment  to  the  ideas).  If  the  plan  did  not  work  as  well 
when  imposed,  this  would  suggest  that  participation  was  useful  not 
just  as  a  method  of  obtaining  good  ideas  but  also  as  a  method  of 
insuring  acceptance  of  or  commitment  to  the  ideas.  The  literature 
is  not  totally  consistent  on  this  point.  Reviews  by  Locke  and 
Schweiger  (1969)  and  Locke  et  al.  (1980)  found  little  evidence 
for  participation  as  a  motivator,  but  Lawler  and  Hackman's  (1969) 
findings  suggest  that  it  can  be  a  motivator  (although  other  inter¬ 
pretations  can  be  made  of  their  findings  as  well) . 

Neider  also  failed  to  separate  the  effects  of  goal  setting 
from  those  of  incentives.  Locke  et  al.  (in  press) concluded  that  both 
may  have  significant  and  independent  effects  on  performance. 

There  are  still  other  possible  interpretations  of  Neider' s  study. 
For  example,  the  difficulty  of  the  goals  in  stores  B  and  D  (and  store 
A  in  the  second  experiment)  may  not  have  been  the  same.  Neider  equated 
the  goals  in  stores  B  and  D  for  subjective  goal  difficulty  (based  on 
ratings)  but  not  for  objective  goal  difficulty  (e.g.,  based  on  the 
absolute  level  of  the  sales  goals  in  relation  to  expected  sales 
based  on  last  year's  sales  and  time  of  year,  etc.).  The  absolute 
goal  difficulty  level  in  store  D  may  have  been  higher  in  that  the 
original  goal  set  for  store  A-  had  to  be  lowered  while  that  for  store 
D  did  not.  Locke,  Shaw,  Saari  and  Latham  (in  press)  found  that 
measures  of  objective  goal  difficulty  and  personal  goals  were  more 
reliably  related  to  actual  performance  than  subjective  goal  difficulty. 


Another  possibility  is  that  the  different  stores  developed 
different  incentive  plans.  Neider  does  not  tell  us  what  the 
actual  rewards  were  in  each  store  for  reaching  the  minimum  and  the 
midrange  goals.  Nor  are  we  told  what  rewards  were  actually  earned, 
since  degree  of  goal  success  was  not  reported.  Since  the  valence  of 
success  was  not  measured,  we  do  not  even  know  whether  the  subjective 
value  of  the  various  incentives  were  equal  across  stores. 

Nhile  the  latter  two  interpretations  might  be  ruled  out  by 
obtaining  further  data  from  the  experimental  records,  the  more 
complex  issue  of  interpreting  an  experiment  with  multiple  indepen¬ 
dent  variables  cannot  be  resolved  conclusively  without  a  more  stringent 
experimental  design  (perhaps  incorporating  fewer  total  variables) . 

Conclusion 

My  goal  in  this  paper  was  not  to  pick  on  Dr.  Neider,  an 
obviously  competent  young  researcher.  Conducting  controlled  field 
studies  is  a  very  difficult  undertaking  ar  d  she  is  to  be  commended 
for  her  work  in  this  area.  My  purpose  here  was  to  call  attention 
to  the  important  issue  of  interpreting  research  findings  and  to  show 
that  the  same  rigor  and  logical  scrupulousness  must  be  used  in  the 
interpretation  as  in  the  design  and  analysis  of  studies. 

As  noted  earlier  Neider  is  not  the  only  researcher  who  can  be 
accused  of  a  questionable  interpretation  of  a  study  of  participation. 
Many  such  studies  are  discussed  in  Locke  and  Schweiger  (1979). 

Another  was  published  more  recently  by  Richter  and  Tjosvold  (1980)  . 
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Their  experiment  purported  to  measure  the  effects  of  student 
participation  on  student  attitudes  and  motivation.  However, 
"Teachers  in  the  student  participation  condition  learned  (during 
training)  about  realistic  goal  setting,  active  listening,  leading  a 
group  without  pre-determined  goals,  asking  questions  and  encourag¬ 
ing  others  to  respond,-  and  constructive  brainstorming"  (p.76). 
While  the  positive  effects  of  the  treatment  were  attributed  to  the 
effects  of  participation  alone,  clearly  no  legitimate  inferences 
about  single  variables  can  be  made  from  this  study  due  to  the  ex¬ 
tensive  confounding. 

A  useful  antidote  to  the  procedure  of  unjustified  or  arbitrary 
interpretations  of  studies  would  be  for  authors  to  go  through  the 
following  procedures  when  interpreting  a  study: 

(a)  What  is  the  evidence  in  favor  of 
the  interpretation  offered? 

(b)  H  ave  alternative  interpretations  been 
ruled  out?  How? 

While  the  first  criterion  is  often  satisfied  in  social  science 
research,  the  second  is  too  often  ignored.  The  result  is  that 
scientific  progress  is  slowed,  sidetracked  or  halted. 

Journal  editors  might  facilitate  the  process  of  objective 
interpretation  by  requiring  a  separate  section  in  each  manuscript 
titled  "Interpretation"  following  the  "Results"  section  or  by 
requiring  that  interpretation,  based  on  the  above  two  criteria. 


be  the  first  issue  addressed  in  the  "Discussion"  section. 
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